Skip to main content

Quicker Q-Learning in Multi-Agent Systems

Item Preview

texts
Quicker Q-Learning in Multi-Agent Systems


Published 2005
SHOW MORE


Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.


Year 2005
Language English
Book contributor NASA
Collection nasa_techdocs

comment
Reviews

There are no reviews yet. Be the first one to write a review.
SIMILAR ITEMS (based on metadata)
eye
Title
Date Archived
Creator
NASA Technical Documents
by Abdeldayem, Hossain A.; Dowdye, Edward; Jamison, Tracee; Canham, John; Jaeger, Tod
texts
eye 191
favorite 0
comment 0
NASA Technical Documents
by Oberlack, Uwe; Vink, Jacc
texts
eye 121
favorite 0
comment 0
NASA Technical Documents
by Miller, Samuel A.; Misch, Noah J.; Dalton, Aaron J
texts
eye 248
favorite 0
comment 0
NASA Technical Documents
by Kacprzak, M.; Lomuscio, A.; Lasica, T.; Penczek, W.; Szreter, M
texts
eye 149
favorite 0
comment 0
NASA Technical Documents
by Roosen, R. G.; Berg, O. E.; Farlow, N. H
texts
eye 196
favorite 0
comment 0
NASA Technical Documents
by Calvignac, Jacky; Dang, Lisa; Tramel, Terri; Paseur, Lil
texts
eye 275
favorite 0
comment 0